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Preface 


This  report  consists  of  two  papers  dealing  with  optimal  control  of 
systems  with  slow  nonlinearities  modeled  as  singularly  perturbed  systems. 

In  the  method  developed  a  composite  control  is  designed  in  two  stages.  A 
slow  nonlinear  and  a  fast  linear  subproblem  need  to  be  solved. 

The  first  paper  by  Chow  and  Ro koto vie  establishes  stabilizing  and 
near  optimality  properties  of  the  composite  control  in  the  deterministic  case. 
In  the  second  paper  by  Bensoussan  the  same  system  is  considered  with  white 
noise  disturbance  inputs.  The  presence  of  noise  smoothed  the  system  behavior 
and  allowed  a  more  complete  solution  than  in  the  deterministic  case. 
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1.  Introduction 

A  conceptually  appealing  framework  for  simultaneous  stabilization 
and  optimization  of  feedback  systems  consists  in  requiring  that  the  Bellman's 
optimal  value  function  be  In  the  same  time  a  Lyapunov  function.  This  has  been 
elegantly  achieved  In  Kalman's  linear  regulator  theory  as  a  culmination  of 
earlier  efforts  by  Lurie,  Krasovski,  Bellman,  and  many  others.  However,  in 
dealing  with  nonlinear  problems,  the  Lyapunov-Bellman  concept  has  serious 
drawbacks.  One  of  them,  the  notorious  "curse  of  dimensionality,"  is 
frustrating  to  practitioners.  Another  one,  the  question  of  existence  and 
differentiability  of  the  optimal  value  function,  disturbs  the  analytically 
minded.  Similar  difficulties  appear  on  the  Lyapunov  side  because  of  the  lack 
of  general  methods  for  constructing  Lyapunov  functions.  Nevertheless,  the 
optimum  stabilization  continues  to  be  one  of  the  fertile  concepts  stimulating 
the  development  of  numerical  and  analytical  methods  for  nonlinear  regulator 
design  [4-7].  Most  analytical  methods  assume  that  the  linear  part  of  the  system  is 
dominant  and  design  a  linear  regulator  as  a  first  approximation,  to  be  subsequently 
corrected  by  series  expansions  [5,7].  This  approach  is  applicable  to  many 
nonlinear  systems,  but  it  also  has  important  limitations.  First,  it  is  not 
directly  applicable  if  the  linear  part  is  not  dominant,  second,  calculation 
of  expansions  increases  the  dimensionality  difficulties,  and,  third, .ill- 
conditioning  due  to  fast  and  slow  phenomena  remains. 

The  two-time-scale  approach  presented  in  this  paper  avoids  lineari¬ 
zation  and  directly  addresses  the  dimensionality  and  ill-conditioning 
difficulties.  Its  philosophy  can  simply  be  stated  as  follows:  "Design  the 
slow  subsystem  first,  by  assuming  that  the  fast  subsystem  has  already  reached 
its  steady  state.  Then  design  the  fast  subsystem  for  a  set  of  constant  values 
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of  Che  states  of  the  slow  subsystem.  Combine  the  two  designs  by  guaranteeing 
stability  and  near-optimality  properties  of  the  resulting  system  . "  The  method 
proposed  In  [3]  and  developed  here  Implements  this  design  philosophy  on  the 
systems  nonlinear  In  slow  variables  and  linear  in  fast  variables  and  control. 

The  class  of  systems  considered  is  assumed  to  be  in  the 
standard  singular  perturbation  form  exhibiting  explicitly  a  parameter  u,  which 
can  be  interpreted  as  the  order  of  magnitude  of  the  ratio  of  the  slow  and 
fast  state  speeds.  Although  this  form  simplifies  the  definition  of  the  sub¬ 
systems,  the  paper  does  not  require  any  familiarity  with  singular  perturbation 
techniques.  The  slow  and  fast  subsystems  can  be  considered  as  postulates 
whose  validity  is  subsequently  demonstrated  by  the  properties  of  the  actual 
system  controlled  by  the  proposed  composite  control.  Since  the  proofs  of 
these  properties  are  elementary  and  make  use  of  only  Bellman's  principle  of 
optimality  and  Lyapunov-type  arguments,  the  paper  can  be  read  with  no  more 
than  a  basic  background  in  control  theory.  The  steps  of  the  design  procedure 
are  presented  on  a  simple  example.  The  method  of  this  paper  is  radically 
different  from  the  finite  interval  trajectory  optimization  results  of  [8,9] 
because  of  the  stability  and  boundedness  requirements  fundamental 
in  infinite  time  problems,  which  require  feedback  solutions. 
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2.  Full  Problem 

The  problem  considered  Is  to  optimally  control  the  nonlinear  system 

x  ■  a^(x)  +  A^(x)z  +  B^(x)u,  x(0)  •  xq  (2.1a) 

UZ  ■  a2(x)  +  A2(x)z  +  B2(x)u,  z(0)  -  zq  (  .lb) 

with  respect  to  the  cost  function 

00 

J  ■  /  [p(x) + s’ (x)z  + z'Q(x)z +  u'R(x)u]dt  (2.2) 

o 

where  u > 0  is  the  singular  perturbation  parameter,  x,  z  are  n-,  m-dimensional 
states,  respectively,  u  Is  an  r-dlmensional  control  and  the  prime  denotes  a 
transpose.  Regulator  problems  where  the  system  Is  linear  In  the  control  and 
nonlinear  In  the  state  have  been  considered  earlier  [6].  Here  the  system  Is 
also  linear  In  the  fast  state  variable  z,  as  is  for  example,  the  case  with 
models  of  dc  motors  and  synchronous  machines  [2].  We  make  an  assumption  which 
in  addition  to  differentiability  and  positivity  properties  of  terms  in  (2.1), 
(2.2)  also  guarantees  that  the  origin  is  the  desired  equilibrium. 

Assumption  I:  There  exists  a  domain  DCRa,  containing  the  origin  as  an 
interior  point,  such  that  for  all  x€D  functions  a^,  a2,  A^,  A^  A^,  B^,  B2, 
p,  s,  R,  and  Q  are  differentiable  with  respect  to  x;  a^,  a2>  p,  and  s  are 
zero  only  at  x*0;  Q  and  R  are  positive  definite  matrices  for  all  x€D;  the 
scalar  p  +  s'z  +  z’Qz  is  a  positive  definite  function  of  its  arguments*  x  and  z, 
that  is,  it  is  positive  except  for  x»0,  z  ■  0  where  it  is  zero. 

An  approach  to  the  full  problem  (2.1),  (2.2)  would  be  to  assume  that 
a  differentials  optimal  value  function  V(x,z,u)  exists  satisfying  Bellman's 
principle  of  optimality 

0  *  mintp  +  s  'z  +•  z'Qz  +  u'Ru  +  Vx(a^+A^z+B^u)  +■“  (a2+A2z+B2u)  ] 


(2.3) 
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where  V  ,  V  denote  the  partial  derivatives  of  V. 

X  2 

(2.3)  is 


Since  the  control  minimizing 


"»  ■  *  I  R"1(Bi,x+ u  TO'  <2-*> 

tha  problem  would  consist  in  solving  the  Hamilton- Jacobi  equation 

0.p  +  x'x  +  «'Q«  +  VxU14*1x)+i  VW'^Vl  +  u  Vx82,R'l(BiTi+u  W" 


V(0,0,u)  -  0. 


(2.5) 


This  would  be  a  difficult  task  even  for  well  behaved  nonlinear  systen.  Due  to 
the  presence  of  ~  terms  in  (2.5),  the  difficulties  with  singularly  p<  ^bed 

systems  (2.1)  Increase.  The  method  of  this  paper  avoids  these  diffic _ xes. 

In  contrast  we  take  advantage  of  the  fact  that  as  u-*0  the  slow  and  the  fast 
phenomena  in  (2.1)  separate.  We  do  not  deal  with  the  problem  (2.1),  (2.5) 
directly.  Instead  we  define  two  separate  lower  dimensional  subproblems,  slow 
and  fast.  The  assumption  about  existence  and  differentiability  of  the  optimal 
value  function  is  then  made  only  for  the  slow  subproblem,  while  the  assumption 
for  the  fast  subproblem  is  similar  to  those  made  for  linear  quadratic 
problems.  The  solutions  of  the  two  subproblems  are  combined  into  a  composite 
control  whose  stabilizing  and  near  optimal  properties  are  the  main  subject  of 
the  paper. 


3.  Slow  Subproblem 

Because  of  the  presence  of  u,  system  (2.1)  exhibits  a  "boundary 
layer,"  that  is,  a  fast  transient  in  the  variable  z,  after  whose  decay  both 
x  and  z  vary  slowly  with  time.  Setting  u*0  the  fast  transient  is  neglected, 
that  is. 


r-  <yww»*  .  ' »Jr* 


*S  *  ai(X3)  +  WS  +  WV  XS(0)  *  XC 


0  *  W  +  A2(xs)zs  +  B2(xs)us’ 
and,  since  is  assumed  co  exist, 

W  ‘  -Aj1(*2  +  B2u.) 


(3.1a) 

(3.1b) 

(3.2) 


is  eliminated  from  (3.1a)  and  (2.2).  Then  the  slow  subproblem  is  to  optimally 
control  the  slow  subsystem 


xs  *  W  +  Vxs)u’ 


x  (0)  »  X 

S  0 


(3.2) 


with  respect  to 


where 


J_  *  /  [p  (x  )  +  2s’(x  )u  +u'R  (x  )u  ]dt 

3  OS  OSS  SOSS 

0 


a0  •  ai  -  V2S 
Bo  ■  B!  -  vlS 


po  ■  ?'a'a2la2  +  a2A2"lqi2la2 


ao  '  B2A2*^<qA2^a2  ~T  9> 
Ro  ■  R  +  B2Ai'lQA2lB2- 


(3.4) 


(3.5) 


We  note  that  xg *  0  is  the  desired  equilibrium  of  the  slow  subsystem  (3.3)  for 

all  x  €D,  since,  in  view  of  Assumption  I,  a  (0)  *  0  and  the  integrand- in  (3.4) 
s  o 

is  positive  definite  in  x  and  u  ,  that  is 

s  s 


p  (x  )  +  2s '  (x  )u  +  u'R  (x  )u  >  0,  x  1*0,  u  ?4  0 . 
os  oss  soss  s  s 


(3.6) 


Our  crucial  Assumption  II  concerns  the  existence  of  the  optimal 
value  function  L(xg)  for  the  slow  subproblem  satisfying  che  optimality  principle 


0  ■  min[p  (x  )  +  2s' (x  )u  +  u'R  (x  )u  + L  (a  (x  )+B  (x  )u  ) ] 
u  os  oss  soss  x  o  s  oss 

3 


(3.7) 
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where  denotes  the  derivative  of  L  with  respect  to  its  argument  xg.  The 
elimination  of  the  minimising  control 

u  -  -R^Cs  +i  B'L' )  (3.8) 

3  O  O  l  O  X 

from  (3.7)  results  in  the  Hamilton-Jacobi  equation 


°  -  (p-s'R^s)  +  L(a-BR"X9)  -  j  LER'Vl'  L(0)-0,  (3.9) 

OOOO  X  O  O  O  O  4  X  O  O  O  X 

where,  due  to  (3.6),  p  -s 'R~^s  is  positive  definite  in  D. 

OOOO 

Assumption  II;  For  all  xg€D  equation  (3.9)  has  a  unique  differentiable 
positive  definite  solution  L(x  )  with  the  property  that  positive  constants 

9 

k^,  k^.  kj,  k^  exist  such  that 

Wi  S  -LJc  *  Wi  <3-10) 

<  -L  a  <  k.a'a  .  (3.11) 

3oo  xo  4oo 

Assumption  II  allows  L(xg)  to  be  used  as  a  Lyapunov  function 

guaranteeing  the  asymptotic  stability  of  x  *  0  for  the  slow  subsystem  (3.3) 

s 

controlled  by  (3.8),  that  is  for  the  feedback  system 


K  *  R*l(s  +i  B'L')  -  a  (x  ) .  (3.12) 

S  0  00  O  2  O  X  OS 

It  also  guarantees  that  D  belongs  to  the  region  of  attraction  of  x  *0.  For 

s 

convenience  we  will  take  a  level  surface  L(xg)  ■  cq  to  be  the  boundary  of  D. 

It  is  pointed  out  that  Assumption  II  does  not  guarantee  the  exponential 

stability.  This  would  be  unnecessarily  restrictive  and  would  exclude  some 

3 

common  slow  subsystems  such  as  xg»-xg. 

Conditions  (3.10),  (3.11)  characterize  the  slow  subproblem  solution 
L  by  bounding  the  rate  L*LxaQ  at  which  it  decays  to  zero  along  the  trajectories 
of  (3.12).  These  bounds  encompass  a  larger  class  of  nonlinear  systems  chan 


do  some  more  common  conditions  based  on  exponential  stability  of  linearized 


models  [5,7].  When  the  solution  L  of  the  &  :v  subproblem  is  known,  conditions 
(3.10),  (3.11)  are  readily  verifiable.  This  is  how  they  are  used  in  our  two 
stage  design.  We  first  solve  the  slow  subproblem  by  one  of  the  existing 
methods,  taking  advantage  of  the  fact  that  its  dimensionality  Is  lower  than 
that  of  the  full  problem.  At  the  end  of  this  stage  L  Is  known  and  (3.10),  (3.11) 
are  checked.  If  they  are  satisfied,  we  proceed  to  the  second  stage,  that  Is 
we  solve  the  fast  subproblem. 

4 .  Fast  Subproblem 

To  motivate  the  formulation  of  the  fast  subproblem  we  observe  that  x 

being  predominantly  slow  means  Chat  only  an  0(y)  error  is  made  by  replacing  x 

with  x  ,  or  vice  versa.  Thus,  when  we  subtract  (3.1b)  from  (2.1b)  we  obtain 
s 

the  system 

u(z-z  )  ■  A_(x) (z-z  )  +  B,(x) (u-u  )  -  uz  (4.1) 

s  2  s  2  s  s 

which  can  be  further  simplified  by  neglecting  the  r.h.s.  0(y)  term  -uz  . 

3 

Defining  z  *  z-z  and  u  *  u-u  the  system  (4.1)  becomes 
rs  t  s 

yzf  »  A2(x)zf  +  B2(x)uf,  zf(0)  •  ZQ-Zg(0).  (4.2) 

Following  a  similar  reasoning  we  define 

00 

■  /  (z^Q(x)zf + ujR(x)u^)dt.  (4.3) 

o 

Now  (4.2)  and  (4.3)  constitute  our  fast  subproblem  for  each  fixed  xED.  It  has 
the  familiar  linear  quadratic  form. 

Assumption  III:  For  every  fixed  xSD 


rank[B2,  B2]  *  m. 


(4.4) 


Alternatively  a  less  demanding  stabillzability  assumption  can  be  made. 

Recalling  also  that  R(x)  >0,  Q(x)  >0  (see  Assumption  I),  we  obtain,  for  each 
xED,  the  optimal  solution  of  the  fast  subproblem 

uf(2^,x)  “  -r”^(x)B2(x)K(x)Z£  (4.5) 

where  K(x)  is  the  positive  definite  solution  of  the  x-dependent  Riccati  equation 

0  »  KA2  +  A^K-KB2R"1B^K  +  Q.  (4.6) 

The  control  (4.5)  is  stabilizing  in  the  sense  that  the  fast  feedback  system 

uzf  -  (A2-B2R'1B^K)zf  -  A 2(x)zf  (4.7a) 

has  the  property  that 

ReX[A2(x)]  <  0,  VxSD.  (4.7b) 


5.  The  Composite  Control 

Compared  to  the  full  problem  (2.1)-(2.5),  the  subproblems  are 
easier  to  solve  due  to  the  fact  that  the  fast  subproblem,  although  parameter 
dependent,  is  a  linear  regulator  problem  and  the  slow  subproblem,  although 
nonlinear,  is  of  a  lower  order  than  the  full  problem.  However,  the  controls 
ug  and  uf  are  applicable  to  the  slow  and  the  fast  subsystems,  respectively, 
which  do  not  exist  in  reality.  Our  goal  is  to  use  ug  and  u^  to 
control  the  actual  full  system  (2.1).  To  accomplish  this  we  now  form  a 
'composite'  control  Ug^^+u^,  in  which  xg  is  replaced  by  x,  and  z ^  by 
z  +  A~^(a„+B,,u  (x)).  Thus  the  composite  control  is 
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l. 

i. 

u,(x,z)  -  Ug(x)  -  R_1B£K(z  +  A"3, (a2-B2ug (x) ) )  j( 

•  -C(*o  +  2  SX>  -  R'1’>2K<j  +  ^1:2)  (5.1) 

where 

a2(x)  -  a2-j  B2R_1(B^r  +8^),  a2(0)  -  0 

V£  -  -(s'  +  Za^K  +  LA^1 

^  -  B1R‘1B^K.  (5.2) 

Note  that  u,  is  Independent  of  u,  which  simplifies  the  design  procedure  when  u 
Is  a  small  but  unknown  parameter. 

For  ufi  to  be  a  meaningful  feedback  control  of  the  system  (2.1),  It 
must  first  of  all  be  a  stabilizing  control.  Furthermore  for  u  to  be  a 
candidate  for  the  optimization  of  (2.2),  the  full  system  (2.1)  controlled  by 
u£  must  result  in  a  bounded  cost  (2.2).  As  the  full  cost  should  approach 

the  cost  of  the  slow  subproblem.  This  would  imply  that  u^  is  a  near-optimal 
control  and  that  the  regulator  problem  is  well-posed.  The  boundedness  and 
near-optlmallty  results  in  the  subsequent  sections  are  new,  while  the  stability 
result  is  essentially  the  same  as  [3],  but  in  a  new  simpler  form. 


6.  Stability 

The  full  system  (2.1)  controlled  by  the  composite  control  (3.1)  is 


x  »  a^  +  A^z  +  B^uc  ■  a^(x)  +  A^(x)z,  x(0)  •  x^ 

\ii  »  a2  +  A2z  +•  B2uc  •  a2(x)  +  A2(x)z,  z(0)  -  zq 


(6.1) 


where 


51  "  al*2  BlR'1(BlLi  +  B2Vl)’  al(0)  "  °» 


(6.2) 
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and  has  the  following  stability  property. 

Theorem  6.1:  If  Assumptions  I-III  are  satisfied,  there  exists  a  u* > 0  such 
that  the  equilibrium  x»0,  z  ■  0  of  system  (6.1)  is  asymptotically  stable  for 
all  u€  (O.w*] . 

Proof:  Introducing 

zf  ■  z  +  *  zo  +  “  *fo  (6*3) 

and  F(x)  ■  (A^a^) x,  we  rewrite  (6.1)  as 

x  -  aQ  +  A^,  (6.4a) 

uif  •  uF(x)ao  +  (A2  + uF(x)A1)zf..  (6.4b) 

i 

Observing  that  (6.4a)  has  the  form  of  the  slow  subsystem  (3.12)  with  the 
additional  forcing  term  A^z^  and  that  (6.4b)  is  an  0(u)  perturbation  of  the 
fast  subsystem  (4.2)  controlled  by  the  fast  control  u^  (4.5),  that  is  of  (4.7a), 
use  the  sum  of  the  slow  and  the  fast  Lyapunov  functions 

u(x,zf,u)  *  L(x)  +  auz’K (x)zf  (6.5) 

as  a  tentative  Lyapunov  function  for  (6.4)  where  a  is  a  positive  scalar  to  be 
chosen.  Since  L(x)  >0  and  K(x)  >0  in  D,  v  is  positive  definite  for  all  s£D, 
z^r"  and  u>0.  The  proof  consists  in  showing  that  the  time  derivative  v  of  v 
with  respect  to  (6.4)  is  negative  definite.  After  completing  the  squares 
\>  can  be  put  in  the  form 

v  •  -g(x,p)  - y  a?'Q(x)?-  az^M(x,Zj,u)Zg  (6.6) 
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where 


g  -  -Li  -  y'Q_1y/2a 


y  -  A^/  +  2c*uKFaQ 

C  -  zf  -  Q_1y/« 


(6.7) 


M  -  Q/2  +  KB2R  BjK-  y(KFA^  +  A^F'K)  -  uK. 


Using  Che  fact  Chat  x-dependent  quantities  In  g  are  bounded  for  xsD, 


that  is. 


'A^Q"1^!  <  k5,  lA1Q"'LKFl  <  kfi,  4lF'KQ“1KFl  <  k? 


it  «-l 


* 


(6.8) 


and  recalling  that  k,L  l'  <  -L  I  ,  k.i'l  <-L  S  ,  see  (3.10) , (3.11) ,  we  obtain 

1  x  x  xo  j  o  o  xo 

y’Q^y  <  (kj  +  3aukg)LxL^  +  (3aykfi  +  a^2^)!^  <  -oL^  (6.9) 


where 


cr(au)  "  k^(kj  +  3aukg)  +  k^Vayk^  +  a2p2k7) . 


It  follows  from  (6.9)  that 


g  >  -Lxao(l-o/2a) 


(6.10) 


(6.11) 


and  hence,  to  make  g  positive  definite,  it  is  sufficient  to  choose  a>a/2. 
A  convenient  choice  is  to  cake  a  to  be  the  value  of  a  when  ay*  1.  Since  cr 
is  a  monotonlcally  increasing  function  of  ay>0,  this  choice  implies  chat 


8  -  '  2  Vo  >  ° 


VU€  (0, 


(6.12) 


To  complete  the  proof  we  need  to  show  that  M  is  also  positive  definite. 


Noting  that  the  first  two  terms  of  M  are  positive  definite  we  now  establish 
that  they  dominate  the  last  two  terms,  which  are  small  for  y  sufficiently 
small.  Using  the  bounds  (6.8)  and 


*1 
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IkI  -  Ik  il  <  Ik  IIS  +  A.Z.I  (6.13) 

x  x  o  T.  f 

w«  conclude  that  there  exist  positive  constants  and 

M  >  i  (Q  +  KBjlf1^  (6.14) 

holds  for  all  x€D,  all  z^  such  that  I  z^l  <  kg,  and  all  u€  (0,u^]<  Thus  for  all 

u€  (0,u*],  u*  •  min(^-,  i^)  (6.15) 

the  derivative  v  of  v  in  (6.5)  for  system  (6.1),  or,  equivalently,  for  system 
(6.4),  is  negative  definite  and  hence  the  equilibrium  x *  0 ,  z  ■  0 ,  is 
asymptotically  stable. 

From  this  proof  we  can  readily  obtain  an  estimate  of  the  region  of 
attraction  of  xa0,  za0.  A  well  known  estimate  is  the  set  of  points  x,  z 
encompassed  by  the  largest  closed  surface  v(x,z,y)ac*  for  which  v  is  negative 
definite.  To  each  fixed  u€  (0,u*]  there  corresponds  one  such  set  denoted  by 
Sy.  All  sets  contain  all  x€D,  but  differ  in  the  magnitudes  of  z,  because, 
as  it  can  be  Inferred  from  the  above  proof,  the  larger  u  is,  the  smaller  z^  is 
allowed.  Thus  the  set  corresponding  to  the  largest  value  of  u,  that  is  to  u*,  is 
the  largest  set  and  is  denoted  by  S*.  Since  this  set  is  the  intersection  of 
all  sets,  it  can  serve  as  a  common  estimate  for  the  regions  of  attraction 
for  all  values  of  u€  (0,u*].  A  proof  of  this  fact  consists  of  the  calculations 
analogous  to  those  leading  to  (6.6)  through  (6.15),  but  this  time  for  v  with  u 
fixed  at  uau*,  that  is  for  v(x,z,u*),  rather  than  for  v(x,z,u).  Omitting 
these  calculations  we  state  the  result  in  the  form  useful  for  our  subsequent 
analysis. 


kg  such  that 


Corollary  6.2;  Under  the  assumptions  of  Theorem  6.1  there  exist  positive 
constants  u*  and  c*  such  that  the  set 

S*(x,z)  »  {x,z:  v(x,z,u*)  <  c*}  (6.16) 

belongs  to  the  region  of  attraction  of  x*0,  z  ■  0  for  all  u€  (0,u*],  that  Is 
all  trajectories  of  (6.1)  originating  in  S*  at  t  •  0  remain  in  S*  for  all  t>0 
and  converge  to  x“0,  z*0,  as  t 

7.  Boundedness  of  J 

Asymptotic  stability  of  an  equilibrium  at  the  origin  is  not 

sufficient  to  guarantee  that  an  integral  of  the  type  (2.2)  will  be  finite  along 

the  trajectories  asymptotically  converging  to  this  equilibrium.  For  example, 

2  5  2 

when  the  control  u*-x  -x  is  applied  to  the  system  x*x  +u,  then  the  equilibrium 
x*0  of  x*-x^  is  asymptotically  stable.  However  the  solutions  for  x(0)  *  xq  +  0 
are 

x(t)  -  sign(xo)(4t+  (xo)“4)"1/4,  (7.1) 

and  hence  the  cost 

J  -  /  (x4  +  1/2  u2)dt  (7.2) 

o 

is  infinite.  Thus  it  is  not  sufficient  that  our  composite  control  be  only  a 

stabilizing  control.  To  qualify  as  a  candidate  for  near-optimality  u  must  also 

c 

produce  a  bounded  J.  To  show  that  this  is  the  case  we  use  the  following  lemma 
from  [1],  which  is  implicit  in  [4,6]. 

Lemma  7.1;  Suppose  that  system  (2.1)  controlled  by  u(x,z)  has  x«0,  z  ■  0  as 
its  asumptotlcally  stable  equilibrium  for  all  xq,zq€S.  Let  this  fact  be 
established  by  a  positive  definite  Lyapunov  function  q(x,z),  whose  derivative 


q(x,z)  is  negative  definite  In  S.  If  there  exists  a  ball  6  centered  at 
x«0,  z  ■  0  such  that  for  all  x,z€  6, 

p  +  s'z  +  z'Qz  +  u'Ru  +  4  <  0,  (7.3) 

then  the  cost  (2.2)  is  finite  along  all  the  trajectories  which  originate  in  S 
and  is  bounded  from  above  by  q. 

Proof:  Let  t„  be  the  instant  when  a  trajectory  t  originating  from  x_,z_€S 
p  o  o 

enters  the  ball  6  through  x. ,  z  for  the  last  time  and  stays  in  6  thereafter. 

P  P 

The  part  of  the  cost  along  t  over  the  finite  interval  [0 , Cg ]  is  obviously 
finite.  Denying  the  remaining  part  of  the  cost  over  (tg,»)  by  Jg  and 
integrating  (7.3)  from  tg  to  00  we  obtain 

Jg  +  Cq(0,0)-q(Xg,Zg)]  <  0  (7.4) 

which  in  view  of  q(0,0)*0  and  the  fact  that  q(x  ,z.)  is  finite,  proves  that  J 

P  P  I 

is  bounded . 

To  apply  this  lemma  we  substitute  (3.1)  and  (6.3)  for  u,  and  z, 
respectively  into 

ao  on 

J  ■  /  (p + s ?z +  zTQz +  u'Ru  )dt  *  /  f  (x,z)dt  (7,5) 

o  o 

and  rewrite  the  integrand  as 

fc(x,z)  •  -Lxao  -  3{zf  +  Zj(Q+  "  £(x,z£)  *  (7.6) 

where 

s1  •  a  +  KB2r“1(B^Lx  +  B2V1)  +  2(Q  +  (7.7) 

It  is  important  to  note  that  the  dependence  on  z£  in  (7.6)  is  indicated 
explicitly,  that  is,  the  term  LxIq  is  Independent  of  z£.  Furthermore, 
f(x,  -Zj)  >  0  because  f(x,z^)  >  0  for  all  x€  0  and  z^€Ra,  x^O,  z^j*0. 
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Theorem  7.2:  Under  Assumptions  I-IZI,  Che  composite  control  u  produces  a 
cost  .7^  which  is  bounded  from  above  by  4v  for  all  (0,p*]. 

Proof :  From  (6.12)  and  (6.15)  we  obtain 

f (x, zf )  +  4v  <  -f(x,  -af)  <  0.  (7.8) 

From  Theorem  6.1  we  know  that  4v  Is  a  Lyapunov  function  for  system  (6.4)  and 
we  use  It  as  q  In  Lemma  7.1,  which  in  view  of  (7.4)  completes  the  proof. 

8.  Near  Optimality 

The  question  can  now  be  posed  whether  uc>  being  a  stabilizing  control 
which  produces  a  bounded  cost,  is  also  near  optimal  in  the  sense  that  as  u  +  0 
the  cost  J,  tends  to  the  optimal  cost  for  u*0,  that  is  the  optimal  cost  L(x) 
of  the  reduced  problem.  This  question  is  answered  by  expressing  as 

Jc(z,x,y)  »  L(x)  +  uV|(x)z  +  uz'K(x)z  +  uJ^(x,z,u)  (8.1) 

where  the  first  two  u-terms  are  suggested  by  the  linear~quadratlc  form  of  the 

fast  subproblem.  If  we  prove  that  remains  bounded  as  u-**0,  this  will 

guarantee  that  J  (x,z,u) -►L(x) . 
c 

Theorem  8.1;  Under  Assumptions  I- III,  the  composite  control  produces  cost 
(8.1)  in  which  remains  bounded  as  u -*•  0 . 

Proof:  Cost  Jc(x,z,u)  of  system  (2.1)  controlled  by  u,  satisfies  partial 
differential  equation 

P  +  s'z  +  z'Qz  +  u'Ru  +  (J  )  (a,  +  A-Z  +  B.u  ) 
cc  exit  lc 

+  (Jc)z(a2  +  A2z  +  B2uc)/U  -  0,  (8.2) 

<Jc(0,0,p)  -  0. 

We  have  shown  in  [3]  that  the  substitution  of  (8.1)  into  (8.2)  and  the  use  of 
(3.9),  (4.6),  and  (5.2),  reduce  (8.2)  to 
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J4x**l  +  *1**  +  ~  J48(a2  +  A2z)  ■  -(V^z  +  z,K2)x(a1  +  A12), 

J4(0,0,u)  -  0.  (8.3) 

This  expression,  and  Che  face  following  from  Theorem  7.1  that  uJ^  is  bounded, 

are  used  in  Che  Appendix  Co  complece  Che  proof. 

In  addicion  Co  Che  near  opcimalicy  of  Che  contposice  concrol.  Theorem  8.1 

also  shows  ehac  Che  full  regulator  problem  is  well  posed  in  Che  sense  ehae  che 

same  cose  resulcs  from  negleccing  u  in  che  syscem  model  and  Chen  applying  Che 

concrol  u  Co  (3.3),  or  firsC  applying  Che  concrol  u  Co  (2.1)  and  Chen 
s  c 

negleccing  u. 


9.  Two  Scage  Design 

The  seeps  of  Che  proposed  Cwo  scage  design  will  be  presenced  on  a 
simple  example  of  Che  syscem 

x  »  -  x  +  z  (9.1a) 

uz  «  -z  +  u  (9.1b) 


and  che  cose  funccional 


J  -  /  (x6+j  z2+j  u2)dc. 


SCep  1:  The  slow  subproblem 


3  3  , 

x  ■  -  t*  x  +  u 
s  4  s  s 


Js  "  /  (X3  +  US)dt 


conslscs  in  solving  che  Hamilcon- Jacobi  equacion 


T  .  dL  3 

\  sr  ■  x5  • 

3 


(9.2) 


(9.3) 

(9.4) 


(9.5) 


L(0)  -  0 


which  yields 


l'IV 


2  3  ’ 


5  3 

-  -r  x  . 
4  3 


Step  2:  Testing  the  conditions  (3.10),  (3.11) 
v!  4  x5  1  ^2*1  * 


25.  6  5  6  25  .  6 

-rr  k.x  <  7*  x  <  rr  k.  x  , 
16  3  s  *  4  s  “  16  4s 


we  see  that  they  are  satisfied  by 


kl*k2*4> 


k3  ’  \  -  f' 


Step  3:  The  fast  subproblem 


uzf  -  -zf  +  uf 


.  r  ,3  2.1  2. 

Jf  ■  /  (4  zf  +  4  uf)dt 

o 


is  in  this  case  independent  of  x  and  its  solution  is 


k-£’ 


uf  *  ~zf’ 


uif  -  -2zf. 


(9.7) 

(9.8) 

(9.9) 

(9.10) 

(9.11) 

(9.12) 


Step  4:  The  design  is  completed  by  forming  the  composite  control 

(9.13) 

and  applying  it  to  the  full  system  (9.1).  The  final  feedback  system*  (6.1)  is 


3 

U  ■  -X  -  z 
c 


-  I  X3  +  z 
4 


uz  •  -x  -  2z . 


(9.14a) 

(9.14b) 


It  should  be  noted  that  this  system  could  not  have  been  designed  by  methods 
based  on  linearization,  since  its  linearized  model  at  xa0,  z  •  0  has  a  zero 
eigenvalue.  However,  Theorem  6.1  guarantees  that  the  equilibrium  xa0,  za0 
is  asymptotically  stable  for  u  sufficiently  small. 
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Step  5:  With  the  help  of  Theorem  6.1  and  Corollary  6.2  we  can  further 

analyze  stability  properties  of  the  designed  system  (9.14)  which  is  first 

1  3 

transformed  by  z^»z+yx  into  (6.4),  that  is  into 


x  «  -  |  x3  +  z. 
4  f 


UZf  *  -p  y  Jt5  -  (2  «  u  |  x2)zf. 


The  Lyapunov  function  (6.5)  is 


1  4  .  12 

v  -  y  x  +  au  y  zf 


(9.17a) 

(9.17b) 

(9.18) 


and  to  analyze  its  derivative  (6.6)  we  evaluate  the  bounds  (6.3), 


k5 


.  413  2 

k6  -  3  4  2  X  ’ 


i  ^  /  9x  1  4 
k7  -  4  4  16  3  * 


(9.19) 


They  are  to  be  used  to  find  an  a  guarateeing  that  g  in  (6.7)  is  positive 

definite  for  all  xsD.  In  this  example  the  choice  of  D  is  free,  since  the 

slow  subsystem  is  asymptotically  stable  in  the  large.  Suppose  that  we  are 

1  13 

interested  in  xs  [0,  y] .  Then  kg  k  y ,  and  o  is  obtained  from  (6.10) 

as  a*  o(l) ,  that  is 


+ 1(1+ -1.)  -  Mi 

5l3  8J  4V641  480 


(9.20) 


With  this  a  it  can  be  easily  verified  that 

g  -  J  *6(l-jJj  (1-yf  aux2)2)  >  0  (9.21) 

for  all  xg  (0,  and  all  us  (0,  — ] .  Next  we  find  u.  such  that 

L  0t  X 

M  +  >  0 

for  all  xs  (0,  y]  and  all  us  (0,u^l.  Clearly  u^<y^  an<i  hence  u**~,"£|y 


(9.22) 
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guarantees  that  v  is  negative  definite  for  all  u€  (0  ,  y*],  all  x6  [0,  j] ,  and 
all  We  note  that  in  this  example  there  is  no  bound  on  z^  because  K*0 

and  hence  M  does  not  depend  on  z^.  In  general  a  bound  on  z^  would  be  required 
for  positive  definiteness  of  M.  It  should  also  be  noted  that  for  a  different 
set  of  x,  a  different  u*  would  be  obtained.  The  presented  sequence  of 
conditions  for  v  <  0  is  convenient  when  y  is  a  parameter  at  the  designer's 
disposal.  When  u  is  a  fixed  physical  parameter,  an  alternative  treatment 
of  (9.20),  (9.21),  and  (9.22)  starting  with  u  given,  would  determine  the 
allowed  x  and,  in  general,  z^ . 

Conclusion 

The  proposed  composite  control  circumvents  the  dimensionality  and 
conditioning  difficulties  and  takes  advantage  of  the  two  time  scale  behavior  of 
the  considered  class  of  nonlinear  systems.  In  spite  of  the  singularly  perturbe^ 
form  (2.1),  these  systems  need  not  be  singularly  perturbed,  that  is  y  need  not  be 
small.  Among  the  results  of  this  paper  are  the  specific  bounds  on  y,  which,  as 
the  example  shows,  can  be  0.5  or  larger.  Estimates  of  the  region  of  stability  are 
given  which  depend  on  u,  but  not  on  the  assumption  that  y  ■+* 0 .  The  only  result  that 
remains  restricted  to  y-^0  is  near  optimality.  It  is  conceivable  that  by  a 
similar  development  bounds  on  the  performance  loss  can  be  obtained.  4 Another 
improvement  is  likely  in  relaxing  conditions  (3.10),  (3.11).  There  exist 
successful  applications  of  the  composite  control  when  (3.10),  (3.11)  are 
not  satisfied.  Nonetheless  (3.10),  (3.11)  are  less  restrictive  than  exponential 
stability  conditions  based  on  linearization.  In  the  first  stage  of  the  two- 
stage  design  the  lower  order  nonlinear  slow  subproblem  needs  to  be  solved. 

It  would  be  of  interest  to  develop  a  numerical  method  whereby  along  the  slow 


solution  also  the  local  values  of  the  fast  subproblem  matrix  K(x)  would  be 
generated.  Finally,  the  assumption  that  the  fast  variables  appear  linearly 
avoids  technical  complications,  but  is  not  crucial  for  the  applicability  of  the 
two  time  scale  approach.  Extensions  to  broader  classes  of  systems  are  possible. 
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Appendix 


We  complete  the  proof  of  Theorem  8.1  by  first  rewriting  J  as 

Jc  -  L  +  uV^(zf  +  zg)  +  w(zf  +  zg)'K(zf+zg)  +  uJ4 
»  L  +  uV^z^  +  uz^KZj  +  UJ4 


(Al) 


where 


Z  -  Zf  +  zg, 


V1  -  V1  +  2Kzs’ 


zg  »  -A2  (x)a2(x) 


J  «  J.  +  V.'z  +  z'Kz  . 
4  4  Is  s  s 


(A2) 


With  respect  to  system  (6.4)  satisfies  the  partial  differential  equation 

f(x,z.)  +  (J  )  (a  +L zf)  +  (J  )  (pFa  +  (A, +  uFA  )zf)  «  0  (A3) 

r  cxolt  cz  o  i  1  £ 


where  f  is  given  in  (7.6).  Taking  the  partials  of  Jc  in  (Al)  and  substituting 
into  (A3) ,  we  obtain 

(J4)xx  +  (I4)zzf  «  -(ZfVlx  +  V|F+2z^KF)(5o  +  Al2f)  -  z£Kzf  -  - (A4) 

A  further  substitution 

\  •  (A^)’i(s1  +  A1Lx)  (A5) 


where  s^  is  as  in  (7.7),  makes  it  possible  to  complete  the  squares  like  in 
(6.6).  We  thus  establish  that 

-fl  -  (zp2x  +  LxA^A2  F  +  2zp3F)(aQ  +  A1zf)  +  zf(J3x(a0  +  ) ) zf  ^A6) 

is  bounded  from  above  by 

-[(l  +  c3)/k1+ (c1+c2)/k3]Lxa(j  +  (2  +  c4)z£zf  (A7) 

and  from  below  by 

j  [(l  +  c3)/k2+  (Cj+c^/k^jL^  -  (2^c4)z^zf  (A8) 
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where 

cL  >  IH’H^  ,  ^  F 

c2  >  !H^H2I  ,  H2  -  (Vjj£+  2KF)ao 

-1  <A9) 

c3  >  ih3h3i  ,  h3  -  a^a2 

C4  -  'K+y  (v1x  +  2kf)a,  +  2  K&iS27'*'  * 

From  (6.13)  we  know  chat  K,  and  hence  c^,  remain  bounded  as  u-*0.  Furthermore, 
rewriting  f(x,zf)>0  in  (7.6)  as 

s^zf  >  -Llo  +  z^(Q  +  KB2R’1B2)zf,  (A10) 


and  using  the  fact  that  the  right  hand  side  quantity  is  positive  definite  for 
all  xSD,  Zj€RD,  we  obtain  by  substituting  +A2  (aQ  +  A^z^)  for  z^. 


^iVVVf51  -  -(1  +  2c5/k3)Lxio  +  2c6ZfZf 


(All) 


where 

>  8N8  -KAj  )'(Q  +  KB2R_1B2K)A2  II 
cfi  >  II  A^NA.1  . 


(A12) 


Combining  (A6)  and  (All)  we  conclude  that  there  exists  y  >  0  such  that  is 

bounded  by  |yv|,  which,  by  Lemma  7.1,  proves  that 

00 

J4  -  /  f.dt  •  (A13) 

o 


is  bounded. 
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Abstract 

Composite  control  orginally  proposed  in  a  deterministic  context 
is  generalized  to  the  problem  wich  white  noise  Inputs.  However,  the 
approach  used  here  is  radically  different  from  the  deterministic  approach. 
Presence  of  noise  smoothed  the  system  behavior  and  allowed  a  more  complete 
solution  than  in  the  deterministic  case. 


* 
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INTRODUCTION 

We  study  in  this  paper  a  stochastic  version  of  the  problem  considered 
by  J.  H.  Chow  and  P.  Kokotovic  [2].  Namely,  we  consider 

dx  ■  (c(x)z  +  d(x)  +2p(x)v(t))dt  +JF  dw^ 

dz  -  j(a(x)z+b(x)+2a(x)v(t))dt+v5‘  d»2 

x(0)  •  x,  z(0)  -  z 
00 

J*  _(v(-))  *E  J*  e”Yt[(f(x)  +h(x)z)2  +  v(t)2]dt. 

O 

Chow  and  Kokotovic  have  considered  this  problem  without  driving  white  noises* 

It  turns  out  that  the  introduction  of  the  noises  smoothes  the  system,  and 
allows  to  obtain  a  fairly  complete  solution  of  the  singular  perturbation  pro* 
blem,  without  the  assumptions  made  in  the  deterministic  case*  We  however 
assume  all  function  of  x  sufficiently  smooth  and  bounded,  and  the  discount  Y 
large  enough  (but  fixed) . 

We  write  formally  the  equation  of  dynamic  programming  and  study  its 

asymptotic  expansion*  We  prove  that  all  the  terms  of  the  expansion  are  uniquely 

defined  and  smooth  (depending  on  the  smoothness  assumptions  on  the  coefficients) . 

Then  as  in  Chow  and  Kokotovic  we  consider  a  composite  control  and 

prove  that  it  maintains  the  pay  off  bounded  by  a  constant  independent  of  * . 

From  that  it  follows  thatinf  J€  (v(*))  remains  bounded  as  «-0.  It  is  possible 

x,  z 

from  this  estimate  to  show  that  the  initial  equation  of  dynamic  programming 
has  a  maximum  solution  in  some  Sobolev  space  with  weights  (as  in  Bennoussan- 
Lions  [1]).  However  we  cannot  prove  a  convergence  result  for  the  inf.  What 
we  prove  is  that 
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inf  J*  _(v(.))-u®(x) 

V  *»*  ° 

where  V  is  a  more  restrictive  class  o£  controls  (namely  chose  for  which 

09 

E  j  clz(t)  l2dt  <M,  where  M  is  a  constant  independent  of  c).  Moreover 
u°(x)  is  the  1st  term  of  the  expansion. 

I  would  like  to  thank  P.  Ko koto vie  for  many  fruitful  discussions 
and  suggestions,  and  first  of  all,  for  having  introduced  me  to  the  problem 
and  organized  my  stay  at  C.S.L.  (Coordinated  Science  Laboratory)  where  this 
research  has  taken  place. 
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1.  Setting  of  the  Model 

Let  us  consider  functions  a(x),  b(x),  c(x),  d(x) ,  a(x) ,  j3(x)  satisfy¬ 
ing 


a*  c.  d,  a,  3  smooth  and  bounded,  aj^O,  ajtO.  (1.1) 

Let  w^(t),  w^(t)  be  two  Wiener  processes,  scalar,  standard  and  independent 
from  each  other. 

We  consider  the  stochastic  system  of  the  equations 


one 
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dx  ■  (c(x)z  +  d(x)  +  2p(x)v(t))dt  +v/5"  dw^ 

dz  • -^(a(x)z +b(x) +2a(x)v(t))dt+vT  dw2  (1.2) 

x(0)  ■  x  z(0)  ■  z. 

.00  _Yfc  2 

The  conCrol  v(*)  is  a  non  antidpative  process  such  that  E  Joe”rC|v(t)(  dt  <«. 
We  consider  the  payoff 

00 

J*  2(v(-))  -  E  J'  e’Yt[(f(x)+h(x)z)2  +  v(t)2]dt  (1.3) 

X,  2  O 

where 

Y>0  constant  (1.4) 

f,  h  bounded  smooth  functions  (1.3) 

we  are  interested  in  the  behavior  as  «-*0,  of  the  Bellman  function 

u‘(x,z)  -  inf  J*  (v(.))  (1.6) 

v(.)  x’z 

( 

Formally  we  can  write  the  Bellman  equation  which  is  satisfied  by  u  .  Namely 
-Au*  +  Yu*  -  (f(x) +h(x)z)2  +  inf[v2+u*  2&V+-#  2ov]  + 

V  * 

«  u  • 

+  ux(cz  4*  d)  +  -^<az  +  b) 


or 

<  € 

oru  u  0 

-Au*+Yu*+(bu*  +  -jV -u*(cz+d)  --J  (az+b)  -  (f  +  hz)Z  (1.7) 

The  optimal  feedback  is  given  by 

.  _  «  «(x)u‘(x,z) 

v  (x,z)  --3(x)ux(x,z) - \ -  (1.8) 


L 

ii 

1 


we  will  refer  to  x  as  the  slow  system  and  to  z  as  the  fast  system.  The  slow 
system  is  strongly  nonlinear,  the  fast  system  is  linear  with  coefficients 
depending  nonlinearly  on  x. 
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We  will  not  study  equation  (1.7)  for  general  c.  It  will  be  used  to 
derive  the  expansion.  Rather  we  will  be  Interested  In  considering  (1.6) 
(taking  It  as  a  definition  of  u*)  for  c  saall.  He  will  define  a  Holt  problem 
which  will  be  the  stochastic  control  problem  for  a  reduced  system  (obtained 
formally  after  multiplication  by  *  and  setting  «  ■  0  in  the  equation  of  the 
fast  system).  The  stochastic  control  problem  for  the  reduced  system  will  be 
solved  completely  using  Bellman  equation.  Now  considering 

w*(x,y)  -  inf  J*  _(v(0) 
v(-)€V  x,z 

where  V  is  a  restricted  class  of  control  (see  (5.6)),  namely  the  class  of 
controls  for  which  the  system  (slow  and  fast)  respect  a  growth  condition,  then 
ve  will  be  approximated,  up  to  e  by  the  value  function  of  the  reduced  system. 

2.  Formal  Expansion 

We  look  for  an  asymptotics  of  the  following  form 

u‘<x«z>  "Jo  ,P  i£o  uX(x)  <2-l> 

where  u^  are  functions  to  be  identified. 

For  convenience  we  define 

u*«0  for  iip  +  2,  and  jl<0.  (2^2) 

The  following  formulas  which  are  easily  verified 


30 


•  ?  «P  i  P 

ux  "  p-0  >0  2  Ufx 


U,  OB  _  P+1  /  -  ■  1 

-5.  Z,  ,p  X-Ci  +  D.V”- 

c  p— 1  4-  O'  '  jfrfl 

c  2  pPt2  i  p 
x  p-0  4-1  /-l,x 


zu 


P+2 


2  .  «  X.  4z*u 

p— 1  i-1 


*  .P+1 


-  OB  _P+1  ,  „ 

U*  -  Zn«pXz\P 

XX  p-0  4-  0  4xx 


P-1 


II  ■f'/5o‘1+l><<  +  2>AW2 


zz 


«  p  ^  x  p 

■  *  r  •  /  flu* 


K + <*f  • a  4-1  uJ + PSo  *p  %  z*  <P“5x  ■ +  <*+ 

P+1 

L  «*"*  “ 

P‘ 

p+2  ,  p  it'  (n+1) 


(P“*+“-r)2  *  o2e'2(uJ)2  +  2uJJq  *p‘l  ^  z^upix  +  a(l+  l)u^}) 

a  p+2  .  p  it' (n+1)  _  _ 

+  plo‘Pios  nio  J5o»U-p«-l)(i,uJx  +  “<J+1)ujtl)('5uP-j,» 


(2.3) 


(2.4) 


(2.5) 


+  «(i  -  J  +  l)ujlj+i) 

and  are  used  in  equating  powers  of  in  (1.7).  We  remark  immediate- 

ly  that 


uj  -  0  (2.6) 

We  then  organize  the  calculations  as  follows.  Assume  that  at  some  stage 
p  a  1  we  know 


u 


but  not  u 


P 

0 


and 


u'  for  D*r*p-1  A-r  +  l,...,0 
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then  we  will  successively  compute 

p+1  „P+l  P+l  P 

p+2’  ur+l . U1  »  u0 

The  case  p  » 1  is  slightly  particular.  We  start  with  it.  So  we  compute 
successively  Uj ,  uj,  u®.  Taking  advantage  of  convention  (2.2)  and  (2.6)  we 
consider  the  sums  in  (2.3)  with  r  running  from  0  to  ®  and  l  from  0  to  p+2. 

Therefore  we  can  write  the  (p ,1)  problem  (p  2  0,0  *  i  *  p  +2)  as  follows 


ij\  (n+1) 


•  <*+ 1><*+J>“*.2  +Y“px  +  »'5o  (^x+«0+l)^(^>x 

+  +<a°rl]  -  uzxlm0+2B»iml 


(2.7) 


+  hZt,2^.0 


we  apply  (2.7)  with  p*0s  jJ*2,  which  will  permit  us  to  compute  u*.  We  obtain 


4a*(uh2  -2auJ-h2 


(2.8) 


and  take 


1  a  +J  +  4q^h^ 

l2  *  m2 

4a 


(2.9) 


We  next  compute  u£  by  writing  (2.7)  with  p-0,  £*1.  We  get 

4aU2^U0x  +  aUl^  "  (cuox  +  2bu2  +aui^  *2fh 

from  which  we  deduce  as  an  affine  function  of  Uq,  namely  (noting 
4a2 u J  -  a  -v£2  +  4<*2h2  -  4) 

^  0^(0-40^12)  +  2fh  +  Ibuj 


(2.10) 


(2.11) 
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We  next  obtain  the  equation  for  u^.  It  comes  from  (2.7)  with  p«0,  0.  We 

obtain 


0  °j_/a  0  ^  lx2  rj  0  ,.  1,  -2 

'u0xx+Yu0+(|Ju0x  +  Q(U1)  -tdu0x  +  bul^“^ 


(2.12) 


But  from  (2 . 10)  and  (2 . 11)  we  deduce 


0  1  uSx(oc-0a)+a(2buJ  +  2fh) 

&u0x+Q(Ui“  - Z - 


(2.13) 


therefore  from  (2.12) 

0  0  (u0x)2(aC"^)2  “2(2buJ  +  2fh)2 

"U0xx  +YuS  +  -^T2 - + - i - +ff<2bu2  +  2fh>  <ac-^)“?x  -  du2x 


buj?  (c-tofti^)  b(2fh+2buh  , 

2 - a - mf^ 


■u^^  +Yuq  +  u^[a(cb-ad)  +  4ha(f  (orc-pa)  +  hOb-dor))  J-^ — -  2  2 

a  +  4a  h 

+  (u°  \2  i^c-pa)2  „  lfa-bh)2 

'•Ox'  2  .  2.  2  2  ,  ,  2.  2 

a  +  4a  h  a  +  4a  h 


(2.14) 


Then  u^  is  solution  of  a  nonlinear  elliptic  equation  which  will  be  studied  in 
the  next  section.  We  can  now  assume  pal,  we  know  u^  for  0<r£p-l,  and 


P  P  P 

p+1  up***ul 


We  compute  successively 


p+i  p+i  p+i  p 

p+2  up+l  •••  ui  u0* 


We  compute  u£J£  by  considering  (2.7)  with  p*p,  i*p+2,  we  obtain 
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n^O^Un+l,x  +  Qt^a+2^un+2^ ^Up-n+l,x  +  Qf(p"n+2^up-ri+2^  *  [cup+l,x +  a(^2)u^21  "° 


i.e. 


p-1 


(P+2)  (4«2uJ  -  a)u^2l  +  <4o*uJ  -  c)u^  1>X  +  ,&(*£*,*  +  «(a+2)u^)  (*£“+lf 

+  a(p-af2)u^_“^2)  »  0 


hence 


p-1 


“E*  ■  t“Jn,*(c-4“1W2)  -  nil(8a»l,x  +  ‘><“+2>'£S>  <p££u,J 


+  o,(p-«2)u'^)l  ^ 


P-1 

pil(n-l"°  ifP"l> 


(2.15) 


Suppose  now  that  we  have  computed 

for  p+2*k*je+l 


P+1 


with  42  2,  and  we  want  Co  compute  4 2 2.  We  consider  equation  (2.7), 


which  we  write  as  follows 


p-1  (n+1) 


-  upte-  (ifl)(l«)uJrt+YuI  +  ;si 

+  a(4-j+l)up'^J)  +2(puPx  +  Qf(ifl)up^J)(puJx  +  o(uJ)  +^U2(|3u4-l,x  +  aiur1) 

■  [cuS-i,x+dus,*+b(irfi)uKi+aiurl1  ■  ° 


from  which  we  deduce 


(4a2u2-a)luP+1  -  (c-to^u^u^  x  +  uP£*(irfL)  (b-2or(0UQX  +  Ofu*)) 

+  up  x(d-2p(Puj^  +  auJ))  +  u*Jxx+(4+l)U+2)u*t2  -YuP 

p-1  4v(n+l)  (3u“  +Qf(j+i)u^J)(^P'“  +a(4-j+l)uP:^) 

n*l  j*0v(4-p+n-l)  jx  J+l  XJ’X  1  J+1 


(2.16) 
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It  remains  to  compute  and  Uq. 


write  (2.7)  for  p  and  4-1.  We  obtain 


-  ui,»*  K+Yui+^t^u^+^r1)(purx+2au2n+1) +(,iui>x+2Q!url)(|iuo,! 

+  <*uP‘n+l)1  +^2(^0r  +  aurl)  +2(^ul,x  +  2Q!ur1)(PU03c  +  aUl)  (2*l7) 

-  [  cu^  +  du^  +  2buP+1  +  au J+1]  -  0 


We  set 


go"dulx'2(3ul  x  +  2aur1)(3u0x  +  QWl)  ■n-lt(3u0x  +  QfUl+1)(iiUl,i  +  2aU2  ) 

p  1X  11  (2.18) 

+  <@ul>x  +  2au2+l)(3u0xa  +QU1  +ul,xx  +  6u3  "YuL 


hence  (2.17)  yields 


^(bu^r1)  -  (cupx  +  2buP+L  +au^+1)  -  gp 
and  by  analogy  with  (2.10)  where  2fh  is  replaced  by  gp, 

p+i  _  unx<c-4apu2)+8p+2bur1 


(2.19) 


We  finally  obtain  the  equation  for  uP.  We  write  (2.7)  for  p,  and  4-0  and 


use  (2.19).  We  obtain 


u0xx+Yu0 ' 2u2  +  n-l<3u0x  +  ““i*  ^  +-rn+l)  +2(pu0x  +  Qur1)(3u0x 

+  cm*)  -  (du^x  +  bui+1)  “  0 


But  by  analogy  with  (2.13) 
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p+1 


q  tH-1  “0*<«-P*> +“<2b^'  ■+*,>> 

e"ox+“ui - ; 


therefore  u^  as  solution  of 


»k*+H+‘y 


2^0x  +  cml^  («c-&a) 


-  d]  *  2uP  +  buP+1  -  2(1^^ 


+  auj) 


a(2bu! 


P+1 

2  +8p^  n  .  n+lwa  p-n  ,  p-n+lN 

A -  '  n2l^u0x  +  aul  )(fiu0x  +QU1  > 


(2.20) 


P 

3.  Study  of  Function  U£ 

The  only  problem  concerns  function  ujj  which  is  solution  of  a  non¬ 
linear  problem.  Set  u*Ug  then  we  can  write  (2.14)  as  follows 


•  u«  +Yu  +  u*^X^  + 


H^u 


(3.1) 


where  X(x),  ^(x),  v(x)  are  given  functions  which  are  bounded  and  that  we  may 
assume  as  many  times  differentiable  as  we  want,  and  Y>0  constant.  We  can 
connect  to  (3.1)  a  stochastic  control  problem  as  follows 


dy  -  (*(y)  +2X(y)v(t))dt+*yr<iw  y(0) -x  (3.2) 

00 

Jx(v(0)  -E  i>oe'YtIv(y)  +v(t)2]dt 

u(x)  -  inf  J  (v(.)).  (3.3) 

v(  * ) 

3.1.  A  Priori  Estimates 

Lamm.  3.1;  Assume  we  have  a  solution  u  of  (3.1)  sufficiently  smooth,  then 


Proof:  Follows  from  the  maximum  principle. 
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VII  »sc. 

L 


Proof:  We  follow  Ladyzhenakaya  Uralt'seva  [1].  Define 

0p(x)  -  exp  -  P(|  x  |2  +  1)1^2 


p  (I  x  |2+1)1/2 


Let  kil.  Set 


u  ■  9(v) ,  9  function  defined  later. 


We  get  from  (3.1)  and 


u*  -9'v'  u"=.9»v,2+?*v"  -  (fv,2  +  'P'v")+a(x)u)ux)  -0 


where 


hence 


a(x,e,p)  ■  X2(x)p2  +  K*)p+Ye  -  v(x) 


v2+i. -0 


The  function  9  will  be  chosen  such  that 


We  next  set 


9*  >  0. 


w  ■  v' 


(3.5) 


(3.6) 


(3.7) 


(3.8) 


T\  -  (w-k)+ 

\  »  i.x|w(x)  >  k}. 

If  is  of  Lebesgue  measure  0,  then  the  result  is  proved.  Let  us  assume  that 
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2 

it  is  of  Lebesgue  measure >0.  We  multiply  (3.7)  by  (2v'Tj)'£  other  the  set 


A^,  hence 


J*  v * 2  +  1  (2v ' Tl)  * dx  -  0. 


(3.9) 


Since  ~i  Is  0  on  the  boundary  of  A^,  we  can  integrate  by  parts 

-  /  p2r-v"  -2"  v'2 +—  1,2v*'idx+.f  ■  — P  —  .... 

*'  'PJ  *  \  <1+ lx  I2) 1/2 


I-V' V2+|,I2V71-0 


CpI  J 


(3.10) 


We  use 


a  o  p  2  p  2Pxp 

•»AkPp"2V'7,<ix"  -JAkPpV,,(2v,71),dx^Ak  (1+|'  ,2^1/2  V"  <2v,‘')dx  (3.1D 

hence 

0"^^t^<2v,^,to+^2v,r‘I^V,2  m^ty  *2vHuh^im  ?'v,2+f')Jdx 

0«JA  P^-w'2  +  (w-k)[-2v»2+2(|tVw2+2  |,'v'w'  -2v'(|,)' 


(3.12) 


2  w'2 

or  using  2v" 


v.cp  • 


ogY  COM  a 

+  i  |2  1/2^"  9*  cpi)3^x 


(l+ix|2): 


^Ak^Ptw'2  +  (w-k)17  *  2(w"k>(|r)  'w2]dx-/A^2(w-k)[2  I*,'  v'w*  -2|,  ^ 


^  .  V*  2Px 

+  2aw — r  + 


cpn  a 
( "  w  +  rai)]^ 


cp*2  (l+|x|2)1/2  *’  ^ 


(3.13) 


We  have 

2  2 

a(x)  » 1  (x)9*  w  +  ^  cp’  v'  +Yu  -  v 

4”  ■  2/.X '9,2w +  l229,y,,v,w +  l2cp,2w’ +^'9' v' +n9"w +  ^9'v" +Ycp*  v'  -  v' 


38 


I'  af -2U  Vv'w  +  2X2<pnw2  +X2?,v'w'  +  H'w  +  M-  v'w  +  £w*  +Yw  -3! 


V 


In  Che  rlghc  hand  side  ox  (3.13)  we  have  terms  involving  w',  namely 

f2 


(w-k)  [2  £  +  \  V  ]v'w'<C[5  (w-k)  ^'2  +  £(w-k)w2  (~2  +  <P'2)  ] 


,2 


(w-k)  |  w*  *C(w-k)[6  J  +iwj 


2  "  ~  ~v"  w  '  6 
Since  w2l  on  A^,  we  majorize  Che  other  cerms  by 


C(w-k)[  I'P’lw2]  +c9(w-k)w3/2 


Going  back  to  (3.13)  we  obtain  choosing  5  small 

„2 


Ja^p^’2  -  2(w-k)(|.')  ’w2J  -/  l50H0^I  +  r2  +  |<p*|)  +CSfjw3/2]d x  (3.14) 


The  constant  depending  on  the  bounds  on  9,  9*,  *5,  but  not  constant  C 


,  1 


r 


Let  M  such  Chat  ||u||  *M.  We  choose 


'P(t)--2M+6M ej  e"S<ld a  ,  q  a  1 


define  t^,  t^  such  that 


T  ^  -sq  1  C  2  -sq  1 

J  esds-^-  /  dsds-~ 

0  o  6e  •  o  2e 


9(t,)  -  -M 


9(t2)  -  M 


-M  *  ?(t)  s  M  for  S  t  «  t,, 


We  have  and  C2  <  ^  since 


.1/2  A 

'  o 


>  a  . 


2. 


I. 

i. 

i. 


so  for  a  choice  of  q  to  be  made  later ,  we  have 
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k  <  tl(q)  <  t  <  tj(q) 


if  -M<{P(t)  <M.  Now 


9'  -  6Me  e 


-t1 


T  m  at 


q-i 


tq_l  -tq 
«F*  «  -6Meqc  e  c 


(|,V  -  -q(q-i)tq'2. 


1 

We  have  9'  >0  9,  9',  9",  jp,  bounded  Let  us  now  choose  q  such  that 


»i^ 


(f.V  >  C(“t'2  +  9’2  +  1 9"  I) 


(3.15) 


or 


q(q-l)tq'2  >C[q2t2q’2  +36M2e2e"2tq  +  6Meqtq"le"t<l] 


which  is  satisfied  for  q  large  enough.  Therefore  we  deduce  from  (3.14) 

J^2!*’2  -  (w-k)  (^,')  'w2]dx <  j A^32(w-k)CCpW3^2dx  (3.16) 

9" 

hence  since  wik  and  -(■^,)'  iC^>0 

kl/2 « % . 
c? 

1/2  ^9  ^9 

Therefore  if  k  >777.  the  set  A.  is  of  measure  0.  This  proves  that  777  is 

a  bound  for  w.  ■ 

3.2.  Existence  and  Uniqueness 

Theorem  3.1:  Assume  that  the  function  a.,  p,  v  in  (3.1)  are  C*,  bounded  with 

bounded  derivatives.  Then  there  exists  one  and  only  one  solution  of  (3.1) 

3 

which  is  C  bounded  as  well  at  its  derivatives. 

Proof:  Let  6(z)  be  a  smooth  function  such  that 


9(z)  ■  z  if  (z)  *k 

I®'  I  <1 

0  bounded  U(z)  |  *Min(  |z  |  ,C) 

We  consider  Che  equation 

-u’,+Yu  +  9(u,)2X2+^' -v  (3.17) 

The  nonlinear  term 

H(x,p)  -e(p)2^2(x) 
is  Lipschicz  in  p,  since 

Hp-20  epX2(x). 

Therefore  there  is  existence  and  uniqueness  of  the  solution  of  (3.17) 

2 

(although  since  d(p)  is  not  convex  equation  (3.17)  does  not  correspond  a 
priori  to  a  control  problem) , 

3  1 

The  solution  of  (3,17)  is  C^«  Now  redoing  the  calculation  of 
Lemma  3. 2, by  virtue  of  the  assumptions  of  9,  once  easily  checks  that  the  same 
estimates  remain  valid.  Now  If  k  is  the  bound  on  |u’l  obtained  in  Lemma  3.2, 
we  see  that  0(u')  «uf.  Hence  the  existence.  Uniqueness  is  the  consequence  of 
the  maximum  principle.  Indeed  if  u,  u  are  two  solutions  then  setting 

v  ■  u  -  u 

we  have  by  difference 

I 

Cb  *  space  of  functions  with  n  derivatives  continuous  and  bounded. 
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r 

r. 


-v*'  +  Yv  +  12  v'(u' +u') +**v' «0 


and  since  u' ,  u*  are  bounded  the  maximum  principle  shows  that  v  *  0.  < 

theorem  3.2.  Under  the  assumptions  of  Theorem  3.1.  (3.3)  holds  and  there 
exists  an  optimal  control. 

Proof:  More  precisely,  we  prove  that  (3.3)  holds  for  the  following  class  of 
a<wi  sslble  controls »  v(t)  is  non  anticlpative. 

Then  the  solution  of  (3.2)  is  defined  in  the  space  K  »  iy Ie  sup  ly(t) |2  <«}, 


VT  finite. 


0£t*T 


By  virtue  of  the  regularity  properties  of  the  solution  of  (3.1), 
the  standard  theory  of  Stochastic  Control  (see  Fleming- Rishel  [4])  yields  the 
desired  result.  The  optimal  control  is  defined  by  the  following  feedback  rule 


u(t)  -  -*(y(t))u'(y(t))  (3.18) 

where  y(t)  is  the  optimal  state.  ■ 

p 

Turning  back  to  function  u^  we  have 

Theorem  3.4.  Assuming  all  functions  of  x  entering  in  (1.2),  (1.3)  C^,  then 

Ug  is  uniquely  defined  by  (2.14)  in  C^,  and  u^,  uj  are  uniquely  defined  and 

1  2  2  2  2  1  0 
C^.  If  the  coefficients  are  Cb  then  u^,  Uj,  u^,  Uq  are  well  defined  and  Cfa. 

If  the  coefficients  are  sufficiently  smooth,  we  can  define  in  a  unique  way 

the  functions  u^  up  to  a  given  index  p. 

2  0  4 

Proof t  Assume  the  coefficients  to  be  (for  instance),  Uq  is  then  in  C^, 

12  12  2  1 
and  Uj  tC^,  u^Cc^.  From  (2.15)  with  r«l,  one  obtains  u^  €cb,  from  (2.16) 

with  r  ■  1,  lm 2  one  obtains  u^  € C^,  from  (2.19)  one  obtains  u2  and  from 

1  2 

(2.20)  Uq  6 .  Clearly  we  can  make  an  induction  argument.  ■ 
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4.  Interpretation  of  the  Limit  Problem 

We  now  give  the  interpretation  of  u°  ■  u  with  respect  to  the  original 
control  problem  (1.2),  (1.3).  This  is  the  reduced  control  problem.  The 
reduced  control  problem  consists  in  setting  after  multiplication  by  e, 

i.e. , 

ax  +  b  +  2arv  ■  0 


hence 


z 


b  -f  2orv 
a 


and  using  this  in  the  slow  system  equation,  we  obtain 

dx  ■  (”  ~(b  +2<*v)  +d  +  20v)dt  +«/5"dw  (4.1) 

x(0)  »  x 

00 

J®(v(-))  -  E  / o  e‘Yt[(f -|(b+2av))2+v2]dt. 

To  see  the  connection  with  (2.14)  and  (3.2)  under  a  suitable  choice  of  k, 
v  we  make  the  following  change  of  control  variable 


v  -  v  £  +  2  X|a.-bh}ah 
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—  (4.2) 

then  after  easy  calculations,  one  can  rewrite  problem  (4.1)  as  follows 


dx  ■  _  bh)oh  +  2  -  (@a  -  crc))dt  +v/5'dw 

/ 

or  as  It  is  easily  verified 

dx  u  a (da  -  be)  +4hot(f  (0a  -  ate)  +  h(dor  -  3b))  (0a  -  ac) 

a2  +  4or2h2  Va2  +  4a2h2 


v+JT 


dw 


(4.3) 


with  a  payoff  functional 


(4.4) 


Jx(v(*))  -E  J*%~Yt(v( t)2  ~ 

°  a^+AaV 

This  is  exactly  the  form  (3.2),  (3.3)  with  the  choices 


(fa  -  bh) 

2  .  .  2.  2 
a  +4or  h 

a  (da  -  be)  +4hg(f  (3a  -  arc)  +  h(da  -  ft?)) 

2  2  2 
ar  +4ah 


3a  -  etc 

7Z7U7 


Therefore  we  may  state. 

Lemma  4.1:  The  function  u»u°  is  the  Bellman  function  of 

“  O  " 

problem,  which  is  obtained  from  (1.2)  as  follows 

dx  »  (cz +d +23v)dt dw^ 

0  •  az  +  b  +  2ov 
x(0)  ■  x 

J®(v(-))  *  E  J%"Yt[(f  +  hz)2  +  v2]dt. 


The  optimal  control  of  this  reduced  problem  is  obtained  by 
feedback 


v,(x) 


(ore  -  3a)  0  . 

2  2  2  u0x  +  2 

a  +  4or  h 


(fa  -  btOoh 
a2  +4o2h2 


1 


We  define  z#(x)  as  the  value  of  z  defined  by  (4. 

the  feedback  control  (4.8)  v  (x),  namely 

s 


1 


s  stands  for  slow. 


(4.5) 

stochastic  control 


(4.6) 

(4.7) 

the  following 

(4.8) 

i 

6)  when  we  apply 


t  Qr~~ 

L  » 


b  +  2orv. 


*s(x) 


-y»*  — >-L 


£  -  ^-(£a  -  M.)h 


i.e. , 


•■w  ■737<h-")‘«'7 


4a  fh  ab 
a2  "  a2 


(4.9) 


We  now  introduce  the  composite  control  as  in  Chow-Kokotovic  [2],  [3]. 


Going  back  to  the  optimal  feedback  (1.8),  we  consider  the  first  term  of  its 


expansion  obtained  from  (2.4),  namely 

vc(x,z)  -  ’(^ox  +  ^J)  -2<*zu2 

and  using  (2.13)  we  obtain 

u°  (£a  -  ac)  -  a(2bu^  +2fh)  , 

„  ..  _  ox  _  z  „ _ J. 


v.(x,z)  — 


Then  we  have 


-  2aau. 


(4.10) 


(4.11) 


L**"*  4.2:  The  composite  control  vc(x,z)  can  be  written  as  follows 


v„(x,z)  -  va(x)  -  2ou2(z-z3). 


(4.12) 


Proof ♦  We  compute  from  (4.11)  and  (4.8) 


vc(x,z)  -  v8(x)  -u°x(^a  -ac)(7f+2j)  -  2au2z  -|(2bu^  +  2fh)  +2 


oh(-fa  +  bh) 


^  4o^u2(Pa  -  ac)u°x  2afh  4a^u2 


-2cm2z  + 


u*  h2 
+  2ab(-  -f  +  \) 
a  AT 


(4. 13) 
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2ob(-  ■“  +  -  2<*b(- 

4  A2  4aZA  Az 


2ab(- 


4a  A 


-V+\)-2ob(- 

4a  A*  4a*A 


A2  -  4a2h2 


4a2  A 


^)-2ab(-  - f-j) 

z  4azA  4a V 


2T*-<l+f)  -  -  —T1 

4a  A  L  tr 


Taking  Into  account  (4.9)  we  obtain  (4.12). 

As  in  Chow-Kokotovic  one  c.n  interpret  the  2nd  term  in  (4.12)  as 

the  optimal  control  for  the  following  control  problem. 

00 

Min  J*q  (h2  z2  +  v ^)  dt 
dzf 

~dt  *  azf +  2ovf 

8.(0)  -  0 


provided  that  we  interpret  z  -  zg  as  z^  (x  being  frozen) , 


5.  Stabilization  Property 

Let  us  prove  now  that,  the  composite  control  maintains  the  initial 
payoff  bounded  as  e~*0,  provided  the  coefficients  are  sufficiently  smooth  and 
Y  is  large  enough,  but  fixed. 

Theorem  5.1:  Assume  the  data  sufficiently  smooth  and  bounded  and  Y  sufficiently 
large  but  fixed.  Choose  in  (1.2)  the  control  to  be  defined  by  feedback  (4.12). 
Then  the  functional  (1.3)  remains  bounded  as  c-*0. 

Proof:  Since  the  coefficients  are  smooth  enough,  we  may  assume  that  the 
functions  v  (x) ,  z  (x)  are  sufficiently  smooth  with  bounded  derivatives. 

Using  feedback  (4.12),  we  consider  the  system 
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dx*cz+d  +  2f*v  -  4afku(z  -  z„)  +V2"dw. 

S  l  S  L 

dz  »  “  [az  +  b  +  2a(vs  -  2ota^(z  m  *s)) Jdt  +vT  dw^ 
x(0)  ■  0,  z(0)  •  z. 


(5 


Setting  z-z^-z^,  we  obtain  a  pair  of  stochastic  processes  x(t),  z^(t) 
solutions  of 


dx  -  (czg  +  d  +  2&vg  +  (c  -  4af*U2>zf)dt  +V5"  dw^ 


dz. 


-  —  A  z -dt  -  z  dx  -  z  dt  +VT  dw-  . 


sx 


sxx 


In  deriving  the  2nd  equation  we  have  used  the  fact  that 


and 


and  by  Ito’s  formula 


az  +  b  +  2av  *  0 
s  s 


/  2  1 
a  -  4a  u^ 


-A 


dz  »  +z  dx  +  z  dt 
s  sx  sxx 


where  z  ,  z  stand  for  the  derivatives  of  z  with  argument  x(t). 

SX  SXX  3 


We  thus  have  simplifying  notation 


dx  •  (A(x)  +l(x)z^)dt  +V^”dw^ 
dzf  »  -  -  Azfdt+ (m(x)  +  6(x)zf)dt 


-  vT  z  dw.  +  */T  dw, 

SX  1  z 


where  A(x),  Y  (x) ,  m(x),  6(x)  are  bounded  functions. 


From  Ito's  formula  we  have 


e  Yt(  |x(t)  I2  +  lzf(t)  I2)  -  Ixl2  +  Iz  -  zg(x)  I2  +/Qe”Y8[-Y(  |x(s)  |2  +  |zf(s)  |2) 

+  (2  +  zsx2)ds]  +J’^e"Y8(2x(s)dx(s)  +2zf  (s)dzf  (s)) 

Set  <P(t)  »  lx(t) I2  +  lt£(t)|2,  we  obtain  by  taking  expectations 

e"YtE*P(t)  +YE  J  e"Ys«P(s)ds  +  2  E  J*  Az2(s)ds  i C  E  Jte"Ys9(s)ds +C  +  |x|2+  |z|2 
°  e  o  1  o 

where  C  is  a  constant  depending  only  on  the  bounds  of  m,  6,  £,  \  and  z  . 

s 

Therefore  if  Y  is  large  enough  but  fixed,  it  follows  that 

00 

E  J*  e  Yt(  lx(t)  I2  +  lz,(t)  |2)dt  <C  independent  of  «, 
o  r 

from  which  it  follows  that 

E  r e’Yt(  l*(t)  I2  +  l*(t)  l2)dt  <®. 
o 

Therefore 

inf  J*  (v(-))£C  independent  of  «.  (5.2) 

v(-)  X>Z 

The  constant  of  course  depends  on  the  initial  condition  x,z.  In  fact  we 

have 

0  *  inf  J*  (v(.))  SC+  Ixl2  +  Iz  I2  (5.3) 

a  v  j  Z 

v(-) 

I 

Lemma  5.1;  When  we  apply  the  composite  control,  the  corresponding  states 
x,  z  satisfy 

E  J  e  Yt( lx(t) +  lz(t) |^)dt s C  (independent  of  e) .  (5.4) 


Proof:  Similar  Co  that  of  Theorem  5.1,  we  apply  Ito's  formula  Co 
e  Yt(lx(t)l^  +  Iz(t)l^)  and  use  the  estimate  already  obtained 

oo 

E  J*  e"Yt(|x(t)  |2+ lz(t)|2)dt<®. 


Let  us  now  introduce 


we(x,z)  -  inf  J®  (V(*)) 
v(.)€V  *’ 


(5.5) 


where 

CO  oo 

V  *  iv(t)  |eJ  e~Yt|v(t)  |2dt  E  J*  e~Yt(  x(t) 
o  o 

(5.6) 

+  |z(t)  |4)dt  *Cj 


The  constants  being  chosen  such  that  the  control  vC(t)  obtained  from  the 
composite  feedback  (4.12)  belongs  to  (5.6)  for  any  e.  This  is  possible  by 
virtue  of  Theorem  5.1  and  Lemma  5.1.  Then  we  can  state 
Theorem  5.2:  We  make  the  assumptions  of  Theorem  5.1,  then 

wg(x,z)  -  u°(x)  pointwise  as  c-0. 

Proof:  Consider 

9e(x,z)  ■  u°(x)  +c(uq(x)  +zuj(x)  +z2u^(x)). 

Let  v(t)  be  a  control  belonging  to  V  and  let  x(t),  z(t)  be  the  corresponding 
solution  of  (1.2)  (depending  of  course  on  «).  By  Ito's  formula 

E<PC(x(t),z(t))e‘Yt  -  ?‘(x,z)+E  jVYs[-Y<P*(x(s),  z(s)) 

o 

+  •  (cz  +  d  +  20v)  +-^  •  -^(az  +  b  +  2<*v)  +^9C(x(s)  ,  z(s))]ds. 


We  can  let  t  and  deduce 


49 


?*(x,z)  -S  J  e”Yt[Yu®  +  e(uQ  +  *uJ  +  z2U2)  -  (cz  +  d  +  2&v) (u°x  +  +  zu 


Lx 


(5.7) 


+  *2«2x>>  •  (az+b  +  2av)(uj  +  2zuj)  *  ^  -  « <»£« +  **1^  +  'Mx*)  *2szu2ldt 
and  using  the  equations  for  Uq,  u*,  u^ ,  we  can  write  (5.7)  as 


QB 

9* (x,z)  -E  J  e‘Yt[(f +hz)*  -  (Puj^  +  ouJ +  2ozu£ +  v)*  +vz  + « (uj + zu J + z*u£) 


2  /fl0  .  1  .  „ _ l  .  .2  .  2  .  1  .  _  1  .  .2  1. 


-  S<cz  +  d)<uL  +  ZUlx  +  z2u21x>  *'i(u0xx  +  zuLcx  +  z2u21xx)  -2ezu2lldt 


(5.8) 


and  since  the  control  belongs  to  V, 


9%x,z)  *  j‘>2(v(-))  +c  « 


therefore  letting  e  *■* 0 


u°(x)  £  lim  w*(x,z) 


(5.9) 


we  now  consider  (5.8)  with  v  equal  to  the  composite  control.  We  deduce  from 
(5.8) 

9*(x,z)  -J*  (vC)  +  0(e)  stw‘(x,z)  +0(e) 

X»a 

hence 

0  -  c 

Un(x)  i  lim  w  (x,z) 


from  which  and  (5.9)  we  obtain  the  derived  result. 


Cone luslon 

In  the  deterministic  problem  considered  by  Chow  and  Kokotovic,  there 
was  no  discount,  and  of  course  no  noise.  They  had  to  assume  that  the  reduced 
control  problem  has  a  solution.  Using  the  composite  control  they  derive 
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under  suitable  assumptions  results  similar  to  those  of  Theorem  5.1  and  5.2 
(stabilization  results) .  We  have  shown  here  that  in  the  stochastic  case 
(with  perfect  information) ,  the  discounted  problem  is  well  posed.  This  allows 
us  to  prove  stabilization  results,  with  the  only  assumptions  that  there  is 
sufficient  smoothness  on  the  data,  and  that  the  discount  factor  is  sufficiently 
large  (but  fixed) .  Our  proof  is  completely  different  from  that  of  Chow- 
Kokotovic . 

Extensions  of  this  work,  can  be  done  in  the  following  directions. 

We  can  use  a  different  scaling  between  noises  and  the  dynamics  of  the  system. 

VT  dw2 

It  seems  reasonable  to  take  -jm  instead  of  v2  dw^  in  the  fast  system 
equation.  One  can  consider  the  vector  case,  i.e.  x,  z  are  vectors  as  in 
Chow-Kokotovic .  It  would  be  nice  to  remove  the  assumption  that  Y  is  sufficient¬ 
ly  large. 

It  would  be  also  interesting  to  study  the  Bellman  equation  for  the 

e  problem. 
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